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PREFACE 


The  work  in  this  report  represents  the  Masters  thesis  of  LCDR  Natanael  Ruiz  Fontes 
carried  out  under  the  guidance  of  Professor  Charles  Therrien  of  the  Naval  Postgraduate 
School. 

This  work  was  motivated  when  Steve  Greineder  and  others  of  NUWC  Code  2121  noticed  a 
significant  signal  distortion  occuring  in  some  cases  when  a  noise  cleaning  algorithm  based 
on  the  Wiener  filter  was  applied  to  some  data  sets.  In  addition,  certain  other  questions 
about  performance  of  the  filter,  such  as  the  typical  processing  gain,  had  been  raised  and 
needed  some  further  attention. 

Rather  than  carry  out  experiments  on  further  empirical  data,  we  decided  to  take  a  primarily 
analytical  approach  to  the  problem.  In  particular,  since  the  answers  to  these  questions 
concerning  the  filter  is  not  available  in  any  of  the  standard  literature,  we  attempted  to 
conduct  an  investigation  of  the  behavior  of  the  basic  Wiener  filter  and  develop  formulas  by 
which  we  could  compute  the  theoretical  limits  of  performance.  This  was  backed  up  by 
results  on  experimental  data  and  would  have  direct  implications  for  the  performance  of  the 
noise  cleaning  algorithm  cited  above. 

The  investigation  of  Wiener  filter  performance  ultimately  led  to  a  simple  generalization  of 
the  filter  where  various  measures  of  performance  could  be  traded  off  against  each  other;  so 
for  example,  signal  distortion  could  be  reduced  with  some  slight  overall  increase  in  residual 
noise  and  mean-square  error. 

The  formulas  in  the  analysis  of  the  HR  Wiener  filter  especially,  are  difficult  to  derive  and 
some  use  was  made  of  a  symbolic  mathematics  program  to  help  in  the  algebraic 
simplification.  Therefore  the  detailed  steps  in  the  derivation  are  not  always  presented,  but 
the  main  formulas  and  steps  are  given.  The  formulas  have  been  checked  carefully  and 
verified  with  experimental  results.  Several  curves  and  graphs  illustrate  the  results. 


I. 


INTRODUCTION 


A.  MOTIVATION  FOR  THE  STUDY 

Data  from  passive  sonar  is  generally  accompanied  by  ambient  noise  arising  from 
shipping  traffic,  marine  life,  wave  motion,  moving  or  cracking  ice  (in  the  Arctic),  and  nu¬ 
merous  other  sources.  The  statistical  properties  of  the  noise  are  variable,  even  direction- 
dependent,  and  have  been  the  source  of  many  studies  and  analyses  [Ref.  1,  2,  3].  Noise 
degrades  sonar  data  collection  and  related  processing  of  the  data  to  extract  information. 

Since  ambient  noise  cannot  be  completely  avoided  when  collecting  real  underwa¬ 
ter  acoustic  data,  it  is  desired,  in  many  situations,  to  remove  the  noise  before  further 
processing.  In  general,  the  signals  of  interest  and  the  ambient  noise  are  non-stationary 
and  their  statistics  are  not  known  a  priori.  Here  we  approach  the  problem  of  removing 
additive  noise  from  a  given  signal  by  using  a  short-time  optimal  filtering  technique.  In 
this  approach  we  assume  that  the  noise  is  stationary  for  the  duration  of  the  signal,  and 
that  the  signal  can  be  assumed  stationary  over  very  short  time  intervals.  We  exploit  this 
feature  and  develop  improved  algorithms  for  removing  the  noise. 

The  work  in  this  report  basically  consists  of  two  parts.  In  the  first  part  (Chapters 
II  and  III),  we  perform  a  detailed  analysis  of  the  Wiener  filter  and  evaluate  its  perfor¬ 
mance  using  three  different  criteria.  For  this  work  we  use  a  simple  signal  model  which 
nevertheless  provides  insight  into  results  for  more  general  types  of  signals.  Both  the  HR 
and  FIR  forms  of  the  filter  are  considered.  In  the  second  part  (Chapters  IV  and  V),  we 
propose  a  new  algorithm  based  on  a  generalization  of  the  Wiener  filter  and  motivated  by 
our  analysis,  that  can  be  applied  to  real  data.  The  implementation  of  the  algorithm  on  a 
short-time  basis  is  similar  to  that  developed  by  Frack  [Ref.  4]  and  we  use  the  same  basic 
structure  of  the  programs.  Results  of  the  application  of  this  algorithm  to  underwater 
acoustic  data  (biologic  data)  are  presented. 
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B.  PROBLEM  DESCRIPTION. 

In  this  section  we  describe  the  problem  of  noise  removal  to  be  addressed  and 
introduce  the  Infinite  Impulse  Response  (HR)  and  the  Finite  Impulse  Response  (FIR) 
forms  of  the  Wiener  filter  used  to  estimate  the  signal  in  noise. 

1.  Problem  Statement. 

We  consider  here  the  problem  of  estimating  a  signal  in  additive  noise.  The  ob¬ 
served  discrete  observation  sequence  is  given  by 

x{n)  =  s(n)  -I-  7?(n)  (I.i) 

where  siri)  is  the  signal  and  T}{ri)  is  the  noise.  The  signal  duration  may  be  anywhere 
from  a  few  miliseconds  to  a  few  seconds  and  is  generally  a  nonstationary  random  process. 
The  noise  is  assumed  to  be  stationary  over  the  entire  observation  interval  and  may  be 
observed  without  the  signal  during  the  early  part  of  the  observation  interval.  That  is, 
the  signal  becomes  non-zero  some  time  after  the  beginning  of  the  observation  interval, 
and  the  precise  time  at  which  it  becomes  non-zero  is  not  known.  Statistics  for  the  signal 
and  noise,  such  as  mean  or  correlation  function,  are  not  known  a  priori  and  must  be 
estimated  from  the  data. 

Let  us  assume  that  the  signal  is  to  be  estimated  by  applying  the  observation 
sequence  to  a  linear  filter  as  shown  in  Figure  1,  and  the  filter  is  designed  to  minimize  the 
mean-square  error 

where  s(n)  =  y{n)  is  the  filter  output.  Then  the  solution  for  this  problem  is  obtained  by 
solving  the  Wiener-Hopf  equation  [Ref.  5],  which  for  the  case  of  a  stationary  signal  has 
the  form 

OO 

Y.  ~  ^)W\  =  Rsx{i)\  0  <  i  <  OO  (1.2) 

z=o 

Here  Rxipj  is  the  correlation  function  of  the  observations,  h{V)  is  the  impulse  response 
of  the  filter,  and  RsxiP)  is  the  cross-correlation  function  between  the  signal  and  the 


2 


x{n)  =  s{n)  +  ■qin) 


m 


y[n]  =  s{n) 


Figure  1.  General  linear  signal  estimation  problem. 


observation  sequence.  Since  the  noise  is  assumed  independent  of  the  signal,  it  is  easy  to 
show  that 

Rx{l)  =  Rs{l)  +  RniP)  (1.3) 

and 

Rsx{l)  =  Rs{l)  (1.4) 

where  Rs{l)  and  Rn{l)  are  the  signal  and  noise  correlation  functions,  respectively.  The 


estimate  for  the  signal  is  then  given  by 

n 

5(n)  =  y{n)  =  ^  h{n-  k)x[k)  (1.5) 

k=—oo 

and  the  mean-square  error  can  be  shown  to  be 

OO 

a^,=RsiO)-^h{l)Rs,{l)  (1.6) 

1=0 

where  again  Rsxil)  =  Rs{l)  for  this  problem. 

When  the  signal  s(n)  is  nonstationary,  as  is  generally  the  case  in  our  application, 
the  optimal  filter  is  time-varying  and  equations  (1.2)  through  (1.6)  take  on  a  slightly 
different  form  [Ref.  5]  involving  the  time  varying  correlation  functions  Rs{ni,n2)  and 
Rx{ni,n2).  Although  these  correlation  functions  could  be  estimated  from  the  data  and 
the  corresponding  time- varying  filter  could  be  computed,  it  is  generally  more  efficient  to 
take  a  block-oriented  approach  where  the  data  is  assumed  stationary  over  short  segments 
and  the  equations  for  a  time-invariant  filter  are  used  to  find  a  filter  that  applies  only  to 
the  data  segment.  Although  we  do  not  do  so  in  this  work,  the  data  segments  could  be 
overlapped  by  some  amount,  and  indeed  when  the  overlap  consists  of  all  but  the  a  single 
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point,  the  result  is  equivalent  to  using  the  time-varying  filter.  Thus  for  purposes  of  our 
work  and  analysis  here,  (1.2)  through  (1.6)  are  the  relevant  equations. 

The  Wiener-Hopf  equation  (1.5)  can  be  solved  for  two  types  of  filters,  the  Infinite 
Impulse  Response  (HR)  filter,  which  is  applied  recursively  to  the  data,  and  the  Finite 
Impulse  Response  (FIR)  filter,  where  the  filtering  process  involves  a  simple  convolution. 
Both  of  these  filters  are  known  as  Wiener  optimal  filters  and  are  described  in  following 
sections.  Later,  we  investigate  the  performance  of  these  two  filters  in  the  case  when  both 
signal  and  noise  are  represented  by  a  first  order  autoregressive  (AR)  model.  This  inves¬ 
tigation  using  a  simple  model  provides  insight  into  the  general  behavior  of  the  Wiener 
Filter,  and  so,  helps  to  predict  its  behavior  for  more  involved  cases. 


2.  Solution  of  The  Wiener-Hopf  Equations: 

The  HR  filter. 


The  HR  Wiener  filter  is  recursive  in  form  and  can  have  the  advantage  of  requiring 
fewer  parameters  than  a  comparable  FIR  form  of  the  filter.  In  many  cases  the  HR  filter 
is  the  only  true  optimal  solution  to  the  problem  and  FIR  filters  are  in  fact  suboptimal 
since  they  do  not  achieve  the  absolutely  lowest  mean-square  error.  Since  the  filter  is 
required  to  be  causal,  the  problem  is  not  straightforward;  it  was  solved  (originally  in  the 
continuous  case)  by  Wiener  using  spectral  factorization  methods  [Ref.  6].  Because  the 
solution  procedure  will  be  referred  to  later  in  this  work,  we  summarize  it  in  this  section. 

The  solution  for  the  IIR  filter  derives  from  observing  that  if  the  observation  se¬ 
quence  is  a  white  noise  process  Rx{l)  =  a^S{l)  then  (1.2)  becomes 

CO 

—  i)h(l)  =  Rsx(i);  0  <i  <  oo  (1.7) 

leading  to  the  simple  solution 


h(l)  = 


-^RsxiO  I  >  0 
0  l<0 


(1.8) 


The  procedure  then  is  to  first  whiten  the  observed  random  process  x  and  then  apply  the 
solution  above.  Thus  we  conceptually  represent  the  optimal  filter  h  zs  &  cascade  of  two 
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filters,  the  whitening  filter  g,  and  the  optimal  filter  h!  for  the  whitened  process.  Both  g 
and  h'  are  required  to  be  causal. 

To  find  the  whitening  process  we  observe  [Ref.  5]  that  the  complex  spectral  density- 
function  of  the  input  process  x  can  be  factored  as 


S.{Z)  = 


(1.9) 


Therefore,  if  we  choose  G{z)  =  l/Hca{z),  we  see  that  the  filter  g  will  have  the  necessary- 
whitening  properties  and  the  whitened  input  process  will  have  variance  Kq. 

The  cross-correlation  between  the  whitened  input  process  and  the  signal  has  a 
complex  cross-spectral  density  function  given  by  Ssx{z) / Hca{z~''-)\  therefore,  the  filter 
corresponding  to  (1.8)  for  the  whitened  process  is  written  as 


Ssx{z) 


/Co  [H,a{z-^) 


(1.10) 


+ 


where  the  notation  [  ]+  means  that  the  resulting  function  corresponds  only  to  the  causal 
part  of  the  quantity  within  brackets.  Finally,  by  cascading  G{z)  and  H' {z)  we  obtain  the 
expression  for  the  optimal  HR  filter 


SsAz)  1 

KoHcaiz) 

[^ca(^-l)J 

+ 

(1.11) 


where  Hca{z)  and  /Co  are  derived  from  the  spectral  factorization  (1.9),  and  Ssx{z)  is  the 
cross  spectral  density  function  for  s{n)  and  x{n). 


3.  Solution  of  The  Wiener-Hopf  Equations: 

The  FIR  filter. 


The  FIR  filter  is  simpler  to  derive.  Since  the  filter  has  finite  length  P,  the  filtering 
equation  (1.5)  can  be  written  as 

p-i 

y{'^)  =  Yh  h{l)x{n  -  1)  (1.12) 

1=0 


We  can  also  directly  apply  (1.2)  noting  that  the  upper  and  lower  limits  need  to  reflect 
the  causality  and  finite  length  of  the  filter.  The  Wiener-Hopf  equation  for  real  process 
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then  becomes 


p-i 

Rx{l  -  i)h{l)  =  Rsx{i);  i  =  0, 1, . . . ,  P  -  1  (1.13) 

and  the  mean-square  error  (1.6)  is  given  by 

p-i 

=  p,(o)  -  x;  h{i)R,s)  (1.14) 

For  the  FIR  filter  it  is  most  convenient  to  write  the  equations  using  vector  nota¬ 
tion.  The  filter  output  is  written  as 


y[n]  =  h^x  (1.15) 

where  h  and  x  are  the  vectors  of  filter  coefficients  and  observations  respectively 


m 

x{n) 

Ml) 

x{n  —  1) 

X  = 

h(p-i) 

x{n  —  P  -f  1) 

Equations  (1.13)  and  (1.14)  can  then  be  written  as 


R^h  —  T^x 


(1.16) 


and 

=  ^5(0)  -  (1.17) 

where  =  E  {x(n)x^(n)}  and  =  E  {s(n)x(n)}.  It  follows  from  (1.3)  that 


Ri  —  Rj  +  R^j  (1.18) 

where  R^  and  R^  are  the  signal  and  noise  correlation  matrices,  and  isx  is  the  first  column 
of  Rj.  This  form  of  the  equations  is  used  in  the  analysis  that  is  presented  later  in  this 
report. 
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C.  ANALYSIS  MODELS. 


The  detailed  analyses  presented  in  this  report  are  based  on  a  first  order  autore¬ 
gressive  (AR)  model  for  the  signal  and  the  noise.  The  signal  s(n)  is  assumed  to  be 
generated  according  to  the  difference  equation 

s(n)  =  0!s(n  -  1)  4-  Wsin)  (1.19) 

where  Ws{n)  is  a  (zero  mean)  white  noise  process  with  variance  and  a  is  a  real  valued 
parameter  with  |q;|  <  1.  The  correlation  function  for  the  signal  is  then  found  to  be 


Rsil)  = 


where  the  signal  power  is  given  by 


1  — 


(1.20) 


(1.21) 


Thus  a  is  seen  to  represent  the  correlation  coefficient  for  the  process  (a  =  R(1)/R(0)) 
with  higher  values  of  a  representing  increased  correlation.  The  complex  spectral  density 
function  for  the  signal  is  the  z-transform  of  the  correlation  function  Rs{l)  and  has  the 
form 


Ss{z) 


^_a|_(l-a^ 
(1  —  az){l  —  OLZ~'^) 


(1.22) 


If  this  is  evaluated  on  the  unit  circle  {z  =  e^’")  we  have  the  power  spectral  density  function 


(1  -  g^) 

1  -I-  0;^  —  2acos{u) 


(1.23) 


For  a  >  0  this  spectrum  has  a  lowpass  character  while  for  a  <  0  the  spectrum  has  a 
highpass  character  (see  Figure  2).  For  a  =  0  the  signal  is  a  white  noise  sequence  and  the 
spectrum  is  fiat. 

The  noise  77(72),  when  it  is  not  white,  is  represented  by  a  similar  model.  The  noise 
satisfies  the  difference  equation 


77(72)  =  777(72  -  1)  -I-  Wr,{n) 


(1.24) 
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frequency  (omega) 


Figure  2.  Power  spectral  density  function  for  real  exponential  correlation  function  (a  >  0 
and  a  <  0). 

where  Wrj(n)  is  a  white  noise  process  independent  of  Ws(n),  and  7  is  real  with  magnitude 
less  than  1.  The  process  has  a  correlation  function  of  the  form 


(1.25) 

where  the  noise  power  is  given  by 

< 

^  1-72 

(1.26) 

and  a  complex  spectral  density  function 

(1.27) 

The  power  spectral  density  function  is  therefore  given  by 

1  -f-  72  _  2jcos(u!) 

(1.28) 
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x{n)  =  s(n)  +  r]{n) 


v{n)  =  y,(n)  +  y„{n) 


- -  H{z)  - 

Figure  3.  General  linear  signal  estimation  problem. 


and  has  a  lowpass  or  highpass  character  according  to  the  sign  of  7.  As  described  earlier, 
the  noise  is  assumed  to  be  added  to  the  signal,  and  the  main  problem  addressed  in  this 
report  is  the  removal  of  the  noise  or  estimation  of  the  signal. 


D.  FILTERING  AND  MEASURES  OF  PERFORMANCE. 

We  have  considered  the  problem  where  we  observe  the  sequence  x(n)  given  in  (I.l) 
where  s(n)  is  the  desired  signal  and  r}{n)  is  additive  noise  uncorrelated  with  the  signal.  It 
is  desired  to  remove  as  much  of  the  noise  as  possible  by  linear  filtering.  Although  we  have 
already  discussed  the  optimal  solution  in  terms  of  mean-square  error,  let  us  temporarily 
put  that  solution  aside  and  formulate  the  problem  more  generally.  We  assume  that  the 
observed  sequence  x(n)  is  applied  to  a  filter  and  the  output  of  the  filter  y(n)  is  an  estimate 
of  the  signal  s(n).  Since  the  filter  is  linear,  we  can  define  components  of  the  output  as 
shown  in  Figure  3.  In  particular  ys{n)  is  the  response  of  the  filter  to  the  signal  alone  and 
yrjin)  is  the  response  to  the  noise  alone.  These  components  permit  defining  the  following 
measures  of  performance: 

Processing  Gain:  The  processing  gain  (PG)  for  the  filter  is  defined  as 


PG  = 


pout 

Pin 


(1.29) 


where  pin  and  pout  are  the  input  and  output  signal-to-noise  ratios  defined  as 


pin  — 


Pout  — 


E{s2(n)}  RM 

EW(ri)}  ■  R,(0) 

(1.30) 

E{yUn)}  _Ry.(0) 

(1.31) 
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The  processing  gain  is  usually  measured  in  dB,  i.e., 


PG(dB)  =  101og,o^2S  (132) 

pin 

Processing  gain  relates  to  the  relative  power  in  the  signal  and  the  noise  but  does  not 
address  accuracy  of  the  signal  estimate. 

Mean  Square  Error:  The  mean-square  error  is  the  expected  value  of  the  squared 
difference  between  the  signal  and  the  filtered  output.  Here  we  use  a  normalized  measure 
of  the  mean-square  error  defined  by 


E{(s(n)-y(ny} 

RM 

where  R^{1)  is  the  correlation  function  for  the  “error”  defined  as  €(n)  =  s{n)  -  s{n)  = 
s{n)  -  y{n). 

When  the  output  of  the  filter  is  equal  to  the  signal  itself  (except  possibly  at  a 
countable  set  of  points)  then  the  mean-square  error  is  equal  to  zero.  The  maximum 
occurs  when  y{n)  is  zero,  where  this  normalized  mean-square  error  has  a  value  of  1. 
Mean-square  error  is  the  quantity  that  is  optimized  in  the  traditional  Wiener  optimal 
filtering  problem  discussed  above. 

Signal  Distortion:  Signal  distortion  (SD)  addresses  what  the  filter  does  to  the 
signal  itself  irrespective  of  the  noise,  and  is  defined  as 


E{s^in)}E{yj{n)}  RsiO)  RyM 


(1.34) 


where  Rsy,{l)  is  the  cross-correlation  function  between  s{n)  and  ys{n).  This  quantity 
takes  on  values  in  the  interval  [0, 1]  and  is  minimized  when  the  signal  is  unchanged  by 
the  filter  {ys(n)  =  s(n)),  i.e.,  there  is  no  distortion  introduced. 


An  ideal  filter  would  form  a  perfect  estimate  for  the  signal  and  completely  elimi¬ 
nate  the  noise.  Such  a  filter  would  have  infinite  processing  gain,  zero  mean-square  error, 
and  zero  signal  distortion.  Since  no  linear  filter  can  achieve  such  ideal  performance, 
we  can  balance  these  measures,  depending  on  the  application,  emphasizing  one  measure 
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compared  to  the  others.  Thus  all  of  the  measures  defined  above  will  have  use  in  the 
sequel. 

E.  OUTLINE  OF  REPORT 

The  remainder  of  this  report  is  organized  as  follows.  Chapter  II  introduces,  anal¬ 
yses  and  discusses  the  performance  of  the  HR  Wiener  filter  described  above.  Chapter 
III  provides  a  similar  analysis  and  discussion  of  the  performance  for  the  FIR  form  of 
the  Wiener  filter.  Chapter  IV  presents  a  filter  based  on  a  new  criterion  that  serves  as  a 
generalization  of  the  FIR  Wiener  filter.  This  filter,  which  we  call  the  extended  optimal 
filter,  provides  a  way  to  improve  the  signal  distortion  with  only  slight  degradation  in  the 
other  performance  measures.  Chapter  V  incorporates  this  technique  in  a  noise  removal 
algorithm  based  on  short  term  filtering  of  the  observation  sequence  and  demonstrates 
its  performance  on  sonar  data.  Chapter  VI  concludes  the  report  with  a  summary  and 
suggestions  for  further  investigation. 
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II.  ANALYSIS  OP  THE  HR  WIENER  FILTER 


In  this  chapter  we  derive  analytical  expressions  for  the  Wiener  HR  optimal  filter 
for  a  signal  in  colored  noise.  The  signal  and  the  noise  are  each  represented  by  a  first 
order  AR  model  (equations  (1.19)  and  (1.24)).  The  expressions  for  the  filter  generalize 
the  results  obtained  in  [Ref.  5]  which  treats  the  white  noise  case  only. 

After  deriving  an  expression  for  the  optimal  filter  we  derive  the  corresponding 
expressions  for  the  measures  of  performance  outlined  in  Chapter  I  and  examine  these 
expressions  as  a  function  of  some  of  the  parameters  describing  the  model.  While  the 
model  expressed  by  (1.19)  and  (1.24)  is  only  of  first  order,  it  exhibits  many  of  the  gen¬ 
eral  characteristics  of  a  more  complicated  model  and  allows  us  to  draw  several  general 
conclusions. 


A.  THE  HR  WIENER  FILTER 

The  general  expression  for  the  optimal  HR  (Wiener)  filter  for  estimating  a  real¬ 
valued  signal  s(n)  in  additive  noise  r][n]  (see  (I.l))  is  given  by  equation  (1. 11).  For  the 
case  of  a  first  order  AR  model,  the  signal  and  noise  complex  spectral  density  functions 
are  given  by  (1.22)  and  (1.27)  respectively.  Since  the  signal  and  noise  are  independent 
with  zero  mean,  it  follows  from  (1.4)  that  the  complex  spectral  density  function  between 
the  signal  s(n)  and  the  observation  x{n)  is  given  by 

a^l  -  a^) 


Ssx{z)  =  Ss{z)  = 


(11.1) 


(1  —  az){l  —  az~^) 

Similarly,  it  follows  from  (1.3)  that  the  complex  spectral  density  function  for  the  obser¬ 
vations  is 


Sxiz)  =  Ss{z)+Sr,{z) 

(7^(1 -7^) 

(1  —  az){\  —  az~^)  (1  —  ^z){\  —  ')z~^) 

_  -  |q|^)(1  +  7^  -JZ-  7Z~^)  -I-  <7^(1  -  |7p)(l  -  az-  ocz~'‘-) 

(1  —  Q'2“^)(l  —  az){l  —  72;“i)(l  —  'yz) 
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By  collecting  terms,  this  last  equation  can  be  written  as 


„  ,  ,  —Az  +  B  —  Az  ^ 

“  D(.) 

(II.2) 

where 

A  =  7(tJ(1  -  a^)  +  acrj(l  -  7^) 

(11.3) 

B  =  crj(l-a^)(l  +  f)  +  (rj(l-7=)(l  +  c^) 

(II.4) 

and 

D{z)  =  (1  -  az~^){l  -  az){l  -  ')z~'^){l  -  72) 

(II.5) 

After  factoring  the  numerator,  we  can  write  (11.2)  as 

_ /iy  ^ 

‘S'x(^)=  {Z  Zia)iz-Zib) 

(11.6) 

where 

1  . 

i 

( 

^ia=  2  (C-VC^) 

(11.7) 

Zlb  =  2  +  n/C^  —  4^ 

(11.8) 

and 

^_B  _  <t|(1  -  a^Xl  +  7^)  +  _  y)(i  +  „2) 

A  0-^1  —  q;^)7  +  (7^(1  —  7^)0; 

(II.9) 

This  last  constant  can  be  expressed  as  a  function  of  a,  7,  and  /)j„,  as 

^  __  [Pm(l  ^^)(1  +  7^)/(l  —  7^)]  +  (1  +  a^) 

[Pinlil  -  -  'f)]  +  a 

(II.IO) 

where  pin  is  the  input  signal-to-noise  ratio 

Pin  —  n 

(11.11) 
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=  < 


(11.12) 


From  the  form  of  the  polynomial  in  (11.2)  and  (II.6),  it  can  be  seen  that  ZiaZu,  =  1,  or 
that  zii,  =  l/zia-  Let  us  denote  the  root  that  is  inside  the  unit  circle  in  the  complex 
plane  as  zi,  i.e., 

f 

Zla  if  \zia\  <  1 

zib  —  Ijzia  otherwise 
Then  from  (II.  10),  we  see  that  Zi  depends  only  on  a,  7,  and  pin- 

We  can  now  factor  the  complex  spectral  density  function  Sx{z)  by  writing  it  in 
the  form 

(1  -  ziz-'^)  (1  -  ZiZ) 


S  (z)  =  —  ■  - 

^  Zi  {1  —  az~'^){l  — 'yz~^)  {1  —  az){l  — '^z) 

In  comparing  this  with  (1.9)  we  can  identify 


and 


Hca{z)  = 


Let  us  now  proceed  to  compute  the  term 


/Co  =  A/ zi 

(1  -  z,z-^) 

(1  —  az~'^){l  —  72:“^) 

^  Ss.{z) 


(11.13) 


(11.14) 


equation  for  the  optimal  filter.  From  (ILl)  and  (IL15),  we 

^sx{.Z^ 


(11.15) 

that  appears  in  (Lll),  the 


+ 

Have 


Hca{z-^) 


<^^(1-|q:P)  {l-az){l-^z) 

(1  —  az){l  —  az~'^)  (1  —  ziz) 


=  ^.(i-i«r) 


2^ 


(1-  (1/7)2:  1) 


2:12:  (1  -  az-'^){l  -  {l/zi)z-^) 

This  can  be  written  in  a  partial  fraction  expansion  as 

Ssxjz)  _  Cl 
Hca{z~^)  (1  -  az 

where 


and 


c2  =  a;(i-|Qp) 


1 

^  (1  -  (l/zi)z->) 

(11.16) 

7  1  -  {1/a-y) 

Zi  (1  -  l/2:ia) 

(11.17) 

7  1  -  (2:1/7) 
zi  (1  -  2:1a) 

(11.18) 
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The  causal  part  of  the  expression  (11.16)  can  now  be  written  as 


1 


H{Z)  =  ir- 


^0 


(11.19) 


C'l 


1  —  az~'‘- 


=  ^  1  ~  Vq^T  <t^(1  -  [Qp)  j/zi 
A  1  —  \/z\OL  1  —  az~^ 

=  £i  -  loip)  tM 

A  a  — Ijzi  1  —  az~'^ 


Finally,  combining  (11.15)  and  (11.20)  we  obtain 


H{z) 


Hcaiz)  JCo  [Hcaiz--^)^ 


Sgx  i^z'j 


+ 


(1  -q;^  ^)(1-7z  \)  £i  O'  -  1/7  (7^(1  -  [gp)  '^/zi 
(1  —  ziz~'^)  A  a  —  1/zi  (1  —  ctz-i) 

(ziazi  —  zij'j  2/,  I  i2\  /  \(^  —  lz~^\ 


2:12: 


-1 


The  final  expression  for  the  Wiener  optimal  HR  filter  can  thus  be  written  as 


where  the  gain  G  is  given  by 


G 


ay  —  1  2 


A  azi  —  1 


<(1  -  IcvP) 


(11.20) 


(11.21) 


(11.22) 


(11.23) 


It  is  interesting  to  note  that  the  filter  places  a  zero  at  2:  =  7  in  an  attempt  to  cancel  the 
additive  noise.  Also,  it  is  important  to  note  that  zi  is  just  a  function  of  the  constant  C, 
and,  therefore,  through  (II.  10),  it  is  just  a  function  of  the  input  signal-to-noise  ratio  pin, 
the  signal  parameter  a,  and  the  noise  parameter  7. 
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The  following  figures  show  the  behavior  of  the  pole  Zi  for  different  situations.  In 
Figure  4,  the  input  signal-to-noise  ratio  {pin)  is  fixed,  but  the  signal  parameter  a  varies 
from  -1  to  +1.  We  have  plotted  the  location  of  the  pole  zi  of  the  optimal  filter  for 
diflferent  values  of  7  (7  =  0  represents  the  white  noise  case).  Notice  that  when  o;  =  7,  the 
filter  places  the  pole  at  the  same  location  {zi  =  7),  since  it  cannot  distinguish  the  signal 
form  the  noise.  Also  observe  that  when  a  =  ±1,  the  pole  is  placed  at  2:  =  ±1,  regardless 
of  the  value  of  7.  In  Figure  5,  the  noise  parameter  7  has  a  fixed  value  of  0  corresponding 


Figure  4.  Location  of  pole  of  the  HR  filter  as  a  function  of  signal  and  noise  correlation 
coefficients  (a  and  7)  for  input  signal-to-noise  ratio  of  0  dB. 

to  white  noise,  and  the  pole  zi  is  plotted  for  different  values  of  the  input  signal-to-noise 
ratio.  Notice  in  Figure  5  that  when  the  input  SNR  is  small,  the  pole  of  the  HR  filter 
approaches  the  signal  parameter  a.  This  can  be  shown  analytically  as  follows. 
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signal  correlation  coefficient  (alpha) 


Figure  5.  Location  of  pole  of  the  HR  filter  as  a  function  of  a  for  7  =  0  and  different 
values  of  the  input  signal-to-noise  ratio. 


First,  from  (II. 10)  we  see  that 


lim  (C)  = 

Pin^^  Oi 


Thus, 


lim  (zi)  = 


1  /  (1  +  a^)  /(I  +  —  4q;2 


a 


1 


2a: 


(2a^)  =  a 


(11.24) 


In  a  similar  manner,  we  can  show  that  for  a  large  input  SNR,  the  pole  approaches  the 
noise  parameter  7.  We  have 

1  +  7^ 


lim  (C) 

pin-^00^  ' 


- 1 - 1 - 1 - 1 - ] _ I _ I _ I _ I _ I 

-1  -0.8  -0.6  -0.4  -0.2  0  0.2  0.4  0.6  0.8  1 

signal  correlation  coefficient  (alpha) 


Figure  6.  Location  of  the  pole  of  the  HR  filter  as  a  function  of  a  for  7  =  -0.5  and 
different  values  of  the  input  signal-to-noise  ratio. 

Therefore 


lim  zl 

pin-^00 


1  /(l  +  7^) 

2  7 

=  7 


(1  +  7^)2  —  47^ 


(11.25) 


This  characteristic  can  also^be  observed  in  Figure  6,  where  we  have  plotted  the  location 
of  the  HR  filter  pole  as  a  function  of  a,  for  different  values  of  the  input  signal-to-noise 
ratio,  and  for  7  =  -0.5.  As  we  will  see  in  a  later  section,  this  behavior  leads  to  a  better 
performance,  in  terms  of  the  processing  gain,  for  smaller  values  of  the  input  signal-to- 
noise  ratio. 
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B.  PROCESSING  GAIN  FOR  THE  HR  FILTER. 

Using  the  first  order  signal  and  noise  model,  we  can  derive  an  analytical  expression 
for  the  processing  gain  (1.29)  for  the  HR  Wiener  Filter.  For  this  we  first  need  the  input 
and  output  signal-to-noise  ratios.  The  input  signal-to-noise  ratio  is  given  by  (1.30)  or 
(II.  11),  while  the  output  signal-to-noise  ratio  is  given  by  (1.31).  Hence,  the  problem  is 
to  find  expressions  for  ^3,^(0)  and  i4^(0)  in  terms  of  a,  7,  and  aj.  To  do  this,  first 
observe  that  the  complex  spectral  density  of  the  output  of  a  linear  system  is  given  by 
[Ref.  5] 

Sy{z)  =  H{z)H{z-^)S,{z)  (11.26) 

where  H{z)  is  the  filter  transfer  function  and  Sx  is  the  complex  spectral  density  function 
of  the  input.  Since  the  correlation  function  is  related  to  the  complex  spectral  density 
function  by  the  inversion  formula 

■^(0  =  Sy{z)z^-^dz 

=  Yl  Residues[6'y(.2:)^:^~^] 

where  the  residues  correspond  only  to  the  poles  within  the  unit  circle,  we  have 


i?j,(0)  =  J^Residues[5'2,(^)2-^] 

=  5Z^®sidues[if(2;)H’(2~^)5x(2)2:~^] 


(11.27) 


To  evaluate  Ry^  (0)  we  use  this  formula  assuming  that  the  input  complex  spectral  density 
function  Sx{z)  is  that  of  the  signal.  Thus  from  (11.26),  (1.22),  and  (11.22)  we  obtain 


Sysiz)z  - 7 

zl  —  ziz  1  —  ziz  (1  —  az  ^){\  —  az) 


(11.28) 


and,  according  to  (11.27)  we  have 


i23,,(0)  =  I{ES,=,,  RES;,^,,  +  RES,=o 

where  the  three  residues  are  given  by 


RES 


Z~Zi 


(z  -  zi){l  -  z^z){l  -  az){l  -  az-^) 


(11.29) 
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-  a^){l  -  7M)(1  -  7^1) 

(1  -  zl){l-alzi){l  -  azi) 


RES^=a  = 


(P{1  -  ^)(1  -  ^z)o‘l{l  -  a^)  _ 

{1  —  ziz-^){l  -  ziz){l  -  Qiz){z  -  ay  ^ 

G^cr^(l-7/o‘)(l-7o:) 

(1-  zi/a)(l-  zia) 


RES2=o 


GH1-7^-^)(1-7z)^!(1-c^^)z  _ 

z{l  -  ziz-'^){l  -  ziz){l  -  az){l  —  az-'^)  ^ 

Thus  combining  these  results,  we  have 


=  0 


z=0 


{1  -  7 / Zi){l  -  7Zi){l  -  Oi^)  ^  {l-7/a){\  -  7a) 


(11.30) 


(1  -  zf){l  -  a/zi){l  -  azi)  (1  -  zi/a){l  -  zia)\ 

In  a  similar  manner  we  see  that  the  output  complex  spectral  density  due  to  the  noise 
alone  is 

1  -  I-7Z  7^) 


Sy(z)  = 


1  —  ziz~^  1  —  ziz'  {1  —  7Z~'^){1  —  7z) 

(1  -  7^) 


Applying  (11.27)  and  evaluating  the  expression  at  the  single  pole  z  =  zi  then  leads  to 

(11.32) 

By  taking  the  ratio  of  (11.30)  and  (11.32)  we  compute  the  output  signal-to-noise 


ratio 


_  (1  -7/21)  (1  -7^1)  ,  (l-7/a)(l-7a)(l-z2) 

Pout  /  \  /-I  \  “T 


(1  -  a/zi)(l  -  azi)  '  (1  -  Zi/q:)(1  -  ziQi)(l  -  0:2)  (11.33) 

Finally,  by  substituting  (II.ll)  and  (11.33)  into  the  definition  (1.29)  we  obtain  the  final 
expression  for  the  processing  gain  as 


PG  = 

■(l-7/zi)(l-7Zi)  ■  (l-7/o:)(l-7o)(l-z?)  ' 

1  —  0:^1 

(1  -  a/zi){l  -  azi)  (1  — •zi/q;)(1  -  Zxa){l  -  a^) 

1  —  72 

(11.34) 


Figure  7  shows  the  theoretical  and  experimental  values  of  the  processing  gain  for 
different  values  of  a,  and  for  7  =  0,  representing  additive  white  noise,  at  a  signal-to-noise 
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ratio  of  0  dB.  Figure  8  shows  the  same  curves,  but  for  7  =  —0.5  (colored  noise)  at  an 
input  signal-to-noise  ratio  of  —10  dB.  The  experimental  curve  was  obtained  by  averaging 
measured  results  for  10  different  trials  for  each  value  of  a,  in  order  to  obtain  a  smaller 
curve  variance.  Notice  that  there  is  very  good  agreement  between  the  theoretical  and 
experimental  values. 


Figure  7.  Comparison  between  theoretical  and  experimental  values  of  processing  gain  for 
the  HR  Wiener  filter  as  a  function  of  a,  for  7  =  0  (white  noise)  and  input  signal-to-noise 
ratio  0  dB. 

Observe  from  Figures  7  and  8  that  the  processing  gain  for  the  first  order  HR  filter 
has  its  maximum  values  at  a  =  ±1.  We  can  examine  the  behavior  explicitly  at  these 
limits  as  follows.  For  a  ^  ±1  the  constant  C  in  (II.IO)  approaches  ±2.  It  follows  then 
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Figure  8.  Comparison  between  theoretical  and  experimental  values  of  processing  gain  for 
the  HR  Wiener  filter  as  a  function  of  a,  for  7  =  -0.5  and  input  signal-to-noise  ratio  -10 
dB. 

from  (II.7)  and  (II.8)  that  the  root  zi  approaches  ±1.  Using  this  result  in  (11.34)  we  find 

_  (1-7)(1-7)(1-^?)  1  _  (1-7)(1-7)(1-2i)(1  +  ^i) 

^  (l-z,)(l-z.)  1-7^  -  (l-^,)(l-^i)(l-7)(l  +  7) 

(l-7)(l  +  ^^i) 

(l-zi)(l  +  7) 

(11.35) 

Figures  9  through  11,  show  the  behavior  of  the  processing  gain  for  different  values 
of  the  input  SNR  (pin),  and  for  7  =  0,  7  =  -.5,  and  7  =  -.999.  Notice  that  in  all 
of  these  cases  the  processing  gain  approaches  infinity  as  a  -)■  ±1.  Intuitively,  we  can 
see  from  (11.22)  that,  for  a  completely  correlated  signal  {a  =  ±1),  the  optimal  HR  filter 
attempts  to  null  the  noise  (by  placing  a  zero  at  7)  while  completely  let  the  signal  pass 
(pole  at  =  q;  =  ±1).  Thus,  while  the  input  signal-to-noise  ratio  remains  finite,  the 
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output  signal-to-noise  ratio  and  thus  the  processing  gain  becomes  infinite. 


Figure  9.  Processing  gain  for  the  HR  Wiener  filter  as  a  function  of  a  for  7  =  0  and 
different  values  of  input  signal-to-noise  ratio. 


C.  MEAN  SQUARE  ERROR  FOR  THE  HR  FILTER. 

An  analytical  expression  for  the  mean-square  error  of  the  Wiener  filter  for  a  first 
order  AR  signal  in  white  noise  is  given  in  [Ref.  5].  Here  we  extend  that  result  to  the 
general  colored  noise  case  and  derive  an  analytical  expression  for  the  MSB  given  by 
equation  (1.33). 

If  the  error  is  defined  as  e{n)  =  s(n)  -  s(n),  then  (1.33)  can  be  written  as 


MSE  =  ^ 


(11.36) 


where  is  the  unnormalized  mean-square  error  or  error  variance.  It  is  shown  in  [Ref. 
5]  that  the  complex  cross  spectral  density  function  between  the  signal  and  the  error  is 
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Figure  10.  Processing  gain  for  the  HR  Wiener  filter  as  a  function  of  o;  for  7  =  —0.5  and 
different  values  of  input  signal-to-noise  ratio. 


given  by 

Substituting  (1.22)  and  (11.22)  then  yields 

<7|(1  - 


s..{z)  =  (1  - G^) 

(1  -  az)(l  -  az  \  1  -  ziz/ 


(11.37) 


(11.38) 


(1  —  az)(l  —  az~^) 

By  the  orthogonality  theorem  of  linear  mean-square  estimation,  the  error  variance  cr^  is 
equal  to  the  cross-correlation  function  Ese(l)  evaluated  at  lag  Z  =  0.  Therefore  from  the 
inversion  formula  for  the  complex  cross  spectral  density  we  have 

O'e  =  Ese(O)  =  ^  Sse{z)z~^dz 

This  integral,  involving  (11.38)  can  be  evaluated  by  residues,  giving 

Residues[S'se(z)2:“^] 


(11.39) 
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Figure  11.  Processing  gain  for  the  HR  Wiener  filter  as  a  function  of  a  for  7  =  -.999  and 
different  values  of  input  signal-to-noise  ratio. 


g'Kl  - 

(1  —  (xz){l  —  az~^) 
ji(.  ^  1  -  7^  ^ 


\-G 


,1-7^ 

1  —  Z\Z 


(z-a) 


=  -.71 -Gj 


Finally,  using  this  result  in  (11.36)  we  obtain 

(11.40) 

Figure  12  compares  the  theoretical  mean-square  error  for  7  =  0  and  pin  =  0  dB 
with  the  averaged  measured  squared  errors  for  the  same  case.  Again,  there  is  a  good 
agreement  between  the  theoretical  and  the  experimental  results.  Figure  13  shows  the 
behavior  of  the  mean-square  error  for  input  signal-to-noise  ratio  of  -10  dB,  0  dB,  and 
10  dB.  Notice  that,  when  a  =  7  =  0,  MSB  =  -H  a^).  For  example,  when  pin  =  0 
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Figure  12.  Comparison  between  theoretical  and  experimental  values  of  mean  square 
error  for  the  HR  Wiener  filter  as  a  function  of  a  for  7  =  0  (white  noise)  and  input 
signal-to-noise  ratio  of  0  dB. 


dB,  the  power  of  the  signal  equals  the  power  of  the  noise,  and  MSE  =  0.5,  as  expected. 
The  same  analysis  for  pin  =  10  dB  results  in  MSE  =  0. 1/1.1  0.1. 

Figure  14  shows  a  case  when  the  noise  is  not  white  (7  =  —0.5).  The  results  are 
qualitatively  similar  to  those  for  the  white  noise  case.  However  the  curves  are  skewed, 
indicating  that  the  worse  performance  in  terms  of  mean-square  error  occurs  when  the 
signal  has  a  correlation  coefficient  equal  to  that  of  the  noise. 


D.  SIGNAL  DISTORTION  FOR  THE  HR  FILTER. 

In  this  section  we  derive  an  explicit  expression  for  the  signal  distortion  (1.34) 
produced  by  the  Wiener  filter  for  the  problem  of  a  signal  in  additive  noise.  As  in  the 
previous  sections,  the  signal  and  the  noise  are  each  represented  by  a  first  order  AR  model. 
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Figure  13.  Mean  square  error  for  the  HR  Wiener  filter  as  a  function  of  a  for  7  =  0  (white 
noise)  and  different  values  of  input  signal- to-noise  ratio. 


Figure  14.  Mean  square  error  for  the  HR  Wiener  filter  as  a  function  of  a  for  7  =  —0.5 
(colored  noise)  and  different  values  of  input  signal-to-noise  ratio. 


resulting  in 

(11.43) 

Then,  by  substituting  (11.43),  (11.30),  and  Rs(0)  =  into  (1.34)  we  can  obtain  an 


expression  for  signal  distortion, 

SD  =  1-  — 

F 

where 

_  (1  -7a)1^ 

[(1  -  zia)_ 


(1 -7Ai)(l -7^i)(l (1  -  7/q!)(1  -  7a) 
(1  -  z?)(l  -  azi){l  -  ajzi)  (1  -  zi/a){l  -  zia) 


(11.44) 


(11.45) 

(11.46) 


Figure  15  shows  the  theoretical  expression  (11.44)  against  the  measured  results  as 


a  function  of  the  correlation  factor  a,  for  white  noise,  with  an  input  signal-to-noise  ratio 
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of  0  dB.  Notice  that  for  values  of  a  close  to  ±1,  i.e.,  for  highly  correlated  signals,  the 
distortion  is  very  low.  This  is  also  apparent  from  Figure  16  when  the  signal  distortion 


Figure  15.  Comparison  between  theoretical  and  experimental  values  of  signal  distortion 

for  the  HR  Wiener  filter  as  a  function  of  cc  for  7  =  0  and  input  signal-to-noise  ratio  0 
dB. 

is  plotted  for  several  different  values  of  the  input  signal-to-noise  ratio.  Figure  16  also 
shows  that  the  signal  distortion  is  lower  for  higher  values  of  the  input  signal-to-noise 
ratio,  because  the  filter  is  able  to  pass  more  of  the  signal.  Since  we  already  know  that 
lim<,_,i2i  =  1,  we  can  evaluate  the  lim^^i  for  (11.46)  and  verify  that  SD  =  0. 

Also,  for  a  =  7  (which  implies  that  H{z)  is  an  all-pass  filter)  the  distortion  is  zero,  as 
expected.  However,  there  is  a  value  for  a,  which  we  shall  call  the  critical  value  where 
the  distortion  reaches  its  maximum  value.  This  value  depends  on 

Figure  17  shows  the  result  for  7  =  -0.5.  Again  the  distortion  peaks  at  specific 
critical  values  of  the  correlation  coefficient  ac-  However,  in  this  case  the  two  values  do 
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Figure  16.  Signal  distortion  for  the  HR  Wiener  filter  as  a  function  of  a  for  7  =  0  (white 
noise)  and  different  values  of  input  signal-to-noise  ratio. 

not  occur  at  locations  symmetrically  distributed  about  the  origin. 

E.  SUMMARY 

The  analysis  of  this  chapter  based  on  the  first  order  AR  signal  and  noise  model 
shows  that  the  optimal  filter  for  estimating  the  signal  is  also  of  first  order.  The  filter  has 
a  zero  at  the  location  7  corresponding  to  the  noise  correlation  coefficient,  and  a  pole  at 
location  zi  which  moves  from  —1  to  +1  as  the  signal  correlation  coefficient  a  moves  from 
— 1  to  +1  (see  Figure  5).  For  low  values  of  the  input  signal-to-noise  ratio  the  pole  zi 
becomes  very  nearly  equal  to  a,  while  for  high  input  signal-to-noise  ratio  Zi  approaches 
7  and  the  filter  becomes  an  all-pass  filter. 

The  processing  gain  (PG)  for  the  optimal  filter  increases  with  increasing  signal 
correlation  and  becomes  infinity  in  the  limit  as  a  approaches  ±1  (perfectly  correlated 


31 


Figure  17.  Signal  distortion  for  the  HR  Wiener  filter  as  a  function  of  a  for  7  =  —0.5 
(colored  noise)  and  different  values  of  input  signal-to-noise  ratio. 

signal).  The  processing  gain  decreases  as  the  input  signal-to-noise  ratio  increases  indi¬ 
cating  that  less  improvement  is  possible  when  the  input  signal-to-noise  ratio  is  already 
high. 

The  normalized  mean-square  error  (MSE)  for  the  optimal  filter  decreases  with 
increasing  correlation  and  becomes  zero  in  the  limit  as  a  approaches  ±1.  It  is  fairly  fiat 
for  most  values  of  a  and  decreases  most  rapidly  as  a  approaches  ±1.  The  normalized 
mean-square  error  (MSE)  also  increases  when  the  input  signal-to-noise  ratio  decreases. 

The  signal  distortion  (SD)  becomes  zero  when  a  =  7,  where  the  filter  becomes 
an  all-pass  filter,  and  when  c.  approaches  ±1,  where  the  signal  is  completely  correlated. 
It  achieves  its  maximum  values  for  moderately  high  values  of  correlation.  In  the  case  of 
white  noise  this  occurs  around  0.8  <  IqJ  <  0.9.  (see  Figure  16). 

The  performance  of  the  filter  for  colored  noise  parallels  that  of  the  filter  for  white 
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noise.  However,  the  symmetry  with  respect  to  a  =  0  is  lost.  When  a  =  7,  the  processing 
gain  goes  to  zero,  the  mean-square  error  achieves  its  maximum  value,  and  the  signal 
distortion  goes  to  zero.  Maximum  values  of  the  signal  distortion  occur  at  two  distinct 
points  above  and  bellow  the  value  of  the  noise  correlation  parameter  7  (see  Figure  17). 
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III.  ANALYSIS  OF  THE  FIR  WIENER  FILTER. 


In  this  chapter  we  develop  expressions  for  the  FIR  form  of  the  Wiener  filter  for 
the  case  where  the  signal  and  the  noise  are,  again,  each  represented  by  a  first  order  AR 
model  (equations  (1.19)  and  (1-24)).  The  chapter  is  divided  into  two  parts.  In  Part  A,  we 
derive  analytical  expressions  for  the  first  order  FIR  Wiener  filter,  i.e.,  a  FIR  filter  with 
only  two  coefficients.  In  Part  B,  we  use  a  more  general  approach  to  derive  expressions 
for  a  filter  of  any  order.  Although  Part  A  treats  a  restricted  particular  case,  it  allows  us 
to  obtain  simple  expressions  for  the  measures  of  performance  as  functions  of  the  model 
parameters.  On  the  other  hand,  the  results  of  Part  B  permit  us  to  plot  the  measures  of 
performance  for  more  general  cases,  where  the  FIR  filter  can  be  of  any  order. 

A.  THE  FIRST  ORDER  FIR  WIENER  FILTER. 

In  Chapter  I  we  introduced  the  matrix  form  of  the  Wiener-Hopf  equation  (1.16) 
for  the  FIR  Wiener  filter  and  the  associated  mean-square  error  (1.17).  For  the  first 
order  FIR  Wiener  filter,  the  vector  of  filter  coefficients  h  has  only  two  elements,  i.e., 
h  =  [h(0)  h(l)]^.  Thus  we  can  obtain  expressions  for  these  two  coefficients  by  solving 
equation  (1.16)  as  follows.  Since  the  noise  is  assumed  uncorrelated  with  the  signal, 
and  Rsx{l)  are  given  by  (1.3)  and  (1.4).  Evaluating  (1.20)  and  (1.25)  for  lag  Z  =  0  and  1, 
and  substituting  in  (1.3)  yields 

Rx (0)  =  0-2  o-J  =  (Jj[l  +  Pin]  (III.1) 

and 

^x(l)  =  (jIcx  +  £7^7  =  alYl  +  apin]  (IIL2) 

Similarly,  from  (L20)  and  (1.4)  we  obtain 

Rsx{^)  =  o]  (IIL3) 
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and 


=  acr^s  (III.4) 
Using  (III.l)  through  (IIL4)  in  equation  (1.16)  results  in 

(Pin  +  1)  {(ypin  +  T)  h(0) 

{fiipin  “1“  "y)  (pin  "h  1)  ^iX) 

Then,  solving  this  matrix  equation  by  inverting  the  matrix,  we  can  express  the  filter 
coefficients  as 

MO)  ^ _ 

_ Pin  _  Pin  1  T)  1  \  1 

_  h{l)  _  (pin  +  1)2  -  {apin  +  7)2  ^  ^  j  ^ 

or. 


(IIL5) 
(IIL6) 

The  transfer  function  for  the  filter  is  then  given  by 

Hiz)  =  h(0)  +  h{l)z-^ 


or 

H{z)  =  h{0){l  -  zqz-^)  (in.7) 

where  the  zero  of  the  filter  is  located  at 


(III.8) 


or 

(III.9) 

Figures  18  and  19  show  the  location  of  the  zero  Zq  of  the  first  order  FIR  Wiener 
filter  as  a  function  of  a  for  different  values  of  7,  and  for  an  input  signal-to-noise  ratio  of  0 
dB  and  -10  dB,  respectively.  From  these  figures  we  can  identify  three  interesting  cases. 
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Figure  18.  Location  of  the  zero  of  the  first  order  FIR  as  a  function  of  the  signal  and 
noise  correlation  coefficients  (a  and  7)  for  input  signal-to-noise  ratio  of  0  dB. 

First,  for  a  =  ±1  and  7  7^  ±1,  and  any  value  of  the  input  signal-to-noise  ratio,  the  filter 
places  the  zero  at  =  Tl-  This  can  be  easily  verified  from  equation  (IIL9).  Secondly, 
when  a  =  ±1  and  7  =  ±1  we  see  that  zq  assumes  different  values.  These  values  can  be 
found  by  setting  7  =  ±1  in  (III.9)  obtaining 

±1-0: 


Pin  ±  1  Pin  ±  ^ 


and  evaluating 


,.  il  —  o  —1  ±1 

zq  =  lim  - - - - - =  lim  - = - 

“-^±1  Pin  ±  1  -  a^pin  T  (X  a-^±l  -2apin  ±  1  2 Pin  +  1 


(III.IO) 


Finally,  when  o  =  7  ^  ±1,  the  filter  places  the  zero  at  zo  =  0,  regardless  the  value  of 
input  signal-to-noise  ratio.  This  seems  intuitively  correct,  since  when  0  =  7  the  filter 
cannot  distinguish  the  signal  from  the  noise,  and  should  become  an  “all  pass”  filter 
(zo  =  0). 
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Figure  19.  Location  of  the  zero  of  the  first  order  FIR  as  a  function  of  the  signal  and 
noise  correlation  coefficients  (a  and  7)  for  input  signal-to-noise  ratio  of  -10  dB. 

Figure  20  shows  the  location  of  the  zero  of  the  first  order  FIR  filter  for  7  =  -0.5, 
and  for  diflferent  values  of  the  input  signal-to-noise  ratio.  The  three  cases  discussed  above 
are  again  evident  from  this  figure. 

1.  Processing  Gain  for  First  Order  Filter. 

To  obtain  an  analytical  expression  for  the  processing  gain  defined  in  (1.29),  we 
first  find  an  expression  for  the  output  signal-to-noise  ratio  (1.31)  of  the  first  order  FIR 
filter.  Starting  with  the  general  expression  [Ref.  5] 

00  00 

~  +  kQ)h(ki)h[ko) 

ko=—00  ki:=z~00 

and  assuming  that  the  signal  alone  is  applied  to  the  filter  we  have 

1  1 

~  ^  +  ko)h{ki)h{ko) 

ko=0  ki=0 
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Figure  20.  Location  of  the  zero  of  the  first  order  FIR  filter  as  a  function  of  a;  for  7  =  —0.5, 
and  for  different  values  of  the  input  signal-to-noise  ratio 


h^l) 

Lh2(0) 


+  1 


oMi)  ^ 

+  OL 

h(0)  ) 


and  by  applying  (III. 8)  we  obtain 

Ry, (0)  =  h\^)Gl{zl  -  2zoa  +  1)  (III.ll) 

By  an  identical  procedure,  we  obtain 

Ry^ (0)  =  h\Q)al{zl  -  2zo-f  +  1)  (111.12) 

Finally,  by  using  (III.ll)  and  (III.12)  in  conjunction  with  (II.ll),  (1-29)  and  (1-31),  we 
can  express  the  processing  gain  for  the  first  order  FIR  filter  as 


PG=  (^0  +  1  ~  2^00^) 
(zg  +  1  -  2zo7) 


(III.13) 
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Figure  21.  Comparison  between  the  theoretical  and  experimental  values  of  the  processing 
gain  for  the  first  order  FIR  filter  for  7  =  0  (white  noise)  and  input  signal-to-noise  ratio 
{pin)  of  0  dB. 

Figures  21  and  22  show  that  there  is  good  agreement  between  the  theoretical  and 
the  experimental  values  of  the  processing  gain  of  the  first  order  FIR  filter,  for  the  case 
when  7  =  0  and  pin  =  0  dB,  and  when  7  =  0.5  and  =  —10  dB,  respectively.  Figure 
23  shows  that  the  processing  gain  is  independent  of  the  input  signal-to-noise  ratio  at 
three  different  points.  The  first  point  occurs  when  o;  =  7,  as  mentioned  before,  when  the 
filter  becomes  an  “all  pass”  filter;  here  the  resulting  processing  gain  is  0  dB.  The  other 
two  points  occur  when  a  =  ±1  and  7  7^  ±1,  where  zq  =  ^1,  and  upon  substituting  this 
result  into  (111.13),  we  find  that  the  processing  gain  becomes  PG\a=±i  =  2/(1  ±7),  i.e., 
independent  of  the  input  signal-to-noise  ratio. 

Figure  23  also  shows  the  general  behavior  of  the  processing  gain  for  the  first 
order  FIR  filter  in  the  white  noise  case,  for  different  values  input  signal-to-noise  ratio. 
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Figure  22.  Comparison  between  the  theoretical  and  experimental  values  of  the  processing 
gain  for  the  first  order  FIR  filter  for  7  =  0.5  (colored  noise)  and  input  signal-to-noise 
ratio  {pin)  of  —10  dB. 


Notice  that  higher  values  of  the  input  signal-to-noise  ratio  results  in  lower  values  of  the 
processing  gain.  Intuitively,  this  happens  because  when  the  input  signal  s(n)  is  strong 
compared  to  the  noise,  the  filter  cannot  make  a  large  improvement.  In  the  limit,  as 
Pin  00  the  processing  gain  goes  to  zero  (except  at  o;  =  ±1). 

2.  Mean  Square  Error  for  First  Order  Filter 

The  (unnormalized)  mean-square  error  can  be  computed  from  (1.14)  using  (III.3), 
(III.4),  (III.5)  and  (III.6)  yielding 

+  Oia^h{l)] 


Pin  +  1-  a^Pin  -  2q;7  -t- 
{Pin  +  1)^  -  {ocpin  +  7)^ 


(III.14) 
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Figure  23.  Processing  gain  for  the  first  order  FIR  filter  as  a  function  of  cr,  for  ^  =  0 
(white  noise),  and  different  input  signal-to-noise  ratio. 

Alternatively,  by  applying  (III.8),  it  can  be  written  as  a  function  of  zq: 

=^s[1-/^(0)[1-q;^o]  (111.15) 

Using  (III.  15)  in  (1.33)  gives  the  normalized  mean-square  error  as 

MSE  =  [1  -  h(0)[l  -  azo]  (III.  16) 

Figures  24  and  25  compare  the  theoretical  and  experimental  values  of  the  nor¬ 
malized  mean-square  error  for  different  values  of  input  signal-to-noise  ratio  and  the  noise 
correlation  parameter  7.  Again,  there  is  a  good  agreement  between  the  theoretical  and 
the  experimental  curves.  Figure  26  shows  the  behavior  of  the  normalized  mean-square 
error  for  different  values  of  input  signal-to-noise  ratio.  We  see  that  for  higher  values  of 
the  input  signal-to-noise  ratio  the  mean-square  error  assumes  lower  values,  as  expected. 
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Figure  24.  Comparison  between  the  theoretical  and  experimental  values  of  the  mean- 
square  error  for  the  first  order  FIR  filter  for  7  =  0  (white  noise)  and  input  signal-to-noise 
ratio  of  0  dB. 


3.  Signal  Distortion  for  First  Order  Filter 

The  signal  distortion  defined  by  (1.34)  is  easily  evaluated.  It  is  easy  to  show  that 
the  cross  correlation  between  input  and  output  of  a  linear  system  is  related  to  the  input 
autocorrelation  function  by  the  convolution  relation  [Ref.  5] 

00 

Rxy{l)  =  h{k)  R^{1  -  k)  (III.17) 

k=—oo 

Hence,  for  our  first  order  filter  we  have 

^...(0)  =  J^h{k)R,{-k) 

A:=0 

=  af(h(0) +  ah(l)) 

=  a^,h{Q){l  -  azo)  (111.18) 
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Figure  25.  Comparison  between  the  theoretical  and  experimental  values  of  mean-  square 
error  for  the  first  order  FIR  filter  as  function  of  a  for  7  =  0.5  (colored  noise)  and  input 
signal-to-noise  ratio  of  — 10  dB. 

Then,  by  applying  (III.  18)  and  (III.  11)  to  (1.34),  we  obtain 


SD  = 

1  2q!,Zo  -f”  Zq 


(III.  19) 


Figure  27  shows  the  theoretical  and  experimental  values  of  the  signal  distortion 
for  different  values  of  a,  and  for  7  =  0  (white  noise)  at  a  input  signal-to-noise  ratio  of  0 
dB.  We  again  observe  good  agreement  between  the  experimental  and  theoretical  results. 
As  in  the  case  of  the  HR  filter,  the  distortion  is  zero  for  o;  =  7  and  a  —  il,  and  achieves 
a  maximum  value  at  a  point  0  <  <  1.  In  Figures  28  and  29,  the  noise  parameter 

7  has  fixed  values  of  0  (white  noise)  and  0.5,  respectively,  and  the  signal  distortion  is 
plotted  for  different  values  of  the  input  signal-to-noise  ratio.  Notice  that  for  lower  values 
of  input  signal-to-noise  ratio  the  signal  distortion  assumes  higher  values.  In  addition. 
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Figure  26.  Mean  square  error  for  the  first  order  FIR  filter  as  a  function  of  a  for  7  =  0.5 
(colored  noise)  for  different  values  of  input  signal-to-noise  ratio. 

notice  that,  regardless  of  the  input  signal-to-noise  ratio,  for  a  =  7  or  for  a  =  ±1,  the 
signal  distortion  is  zero.  This  can  also  be  easily  verified  from  (III.  19). 

B.  PERFORMANCE  MEASURES  FOR  HIGHER  ORDER 
FIR  WIENER  FILTERS. 

For  FIR  filters  of  higher  than  first  order  (P  >  2),  it  becomes  extremely  difficult 
or  impossible  to  derive  analytical  expressions  for  the  filter  coefficients  and  zeros  of  the 
filter  in  term  of  the  basic  parameters  a,  7,  and  />j„.  However  it  is  still  useful  to  be 
able  to  compute  theoretical  values  for  the  processing  gain,  mean-square  error,  and  signal 
distortion  in  these  cases.  Therefore,  in  this  section  we  develop  a  different  approach  to 
derive  expressions  for  these  performance  measures,  which  can  be  plotted  as  a  function 
of  the  various  parameters.  With  these  results,  we  can  compare  the  performance  the  FIR 
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Figure  27.  Comparison  between  the  theoretical  and  experimental  values  of  signal  distor¬ 
tion  for  different  values  of  a  and  for  7  =  0  (white  noise)  at  a  input  signal-to-noise  ratio 
of  0  dB. 

filter  of  any  order  to  that  of  the  optimal  HR  Wiener  filter  when  they  are  applied  to  the 
first  order  signal  and  noise  models  of  equations  (1.19)  and  (1.24). 

1.  Processing  Gain  for  Higher  Order  FIR  Filter. 

The  output  power  of  a  filter  expressed  in  the  form  (1.15)  can  be  written  as 

Rs(0)  =  E{y^(n)} 

=  E{(h"x)  (x-h)} 

=  h^E{xx^}h 

=  h^Rxh  (111.20) 

Therefore,  the  output  power  when  the  signal  and  noise  act  alone  is  given  by 

R3„(0)  =  h"'R,h  (111.21) 
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Figure  29.  Signal  distortion  for  the  first  order  FIR  filter  as  function  of  a  for  7  =  0.5  and 
different  values  of  the  input  signal-to-noise  ratio 

The  processing  gain  produced  by  the  Wiener  filter  can  be  obtained  by  substituting  the 
solution  to  the  Wiener-Hopf  equation  (1.16)  into  (III.24).  Specifically,  from  (1.16)  and 
(1.18)  we  have 

h  =  (R,  +  R^)-^f,  (III.25) 

Since  we  have  all  the  necessary  expressions,  we  can  plot  the  values  assumed  by  the 
processing  gain  (III.24)  for  different  values  of  a,  7,  and 

Figure  30  shows  the  behavior  of  the  processing  gain  as  a  function  of  a  for  7  =  0 
(white  noise)  and  for  different  values  of  input  signal-to-noise  ratio.  Notice  that  the 
processing  gain  has  different  maxima  for  a  =  ±1,  depending  on  the  order  of  the  filter. 
These  maximum  values  can  be  easily  predicted  for  the  white  noise  case  (7  =  0),  as 
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follows.  For  a  =  1  and  7  =  0  the  signal  and  noise  correlation  matrices  (IIL23)  become 


•••  cr^s 

■^2  0 

...  0 

R5  = 

R„  = 

0  al 

...  0 

•  •  •  (^s  . 

0  0 

••• 

(111.26) 


Under  these  conditions  (111.25)  shows  that  h{0)  =  h{l)  =  ...  =  h{P  —  1).  By  applying 
these  results  to  (111.21)  and  (III.22),  we  obtain 


h’’R,h  =  <^2  ;^2(o) 


and 

h^R^h  =  a^P  h^{0) 

Finally,  substituting  these  expressions  into  (III.24)  results  in 


PG  =  P 


A  similar  analysis  shows  that  PG  =  P  also  for  a  =  —1. 

This  result  shows  that  unlike  the  HR  filter,  the  processing  gain  for  an  FIR  has 
a  finite  bound,  which  is  equal  to  the  filter  length.  This  result  can  be  easily  checked  in 
Figure  30  ^  Notice  also  that  in  general,  for  any  value  of  a,  larger  length  filters  produce 
increased  processing  gain. 

2.  Mean  Square  Error  for  Higher  Order  FIR  Filter. 

The  normalized  mean-square  error  (1.33)  for  any  FIR  filter  is  not  difficult  to  obtain. 
Using  (1.15)  and  (I.l)  we  can  write 

E  {(s(n)  -  y(n))^|  =  E  {(s(n)  -  s  -  h  -  h)} 

=  erf  —  2  f  s  -}-  h^  Rs  h  -I-  R,,  h 

^Note  that  the  vertical  scale  in  Figure  30  is  in  dB 
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Figure  30.  Processing  gain  for  the  FIR  filter  of  length  P  for  7  =  0  (white  noise)  and 
input  signal-to-noise  ratio  of  0  dB 


where  the  cross  terms  disappear  because  the  signal  and  noise  are  uncorrelated.  Then, 
substituting  this  expression  in  (1.33)  we  obtain 


MSE  =  1  - 


2  f ,  -  R,  h  -  R„  h 


(III.27) 


The  result  (III.  27)  applies  to  any  FIR  filter.  For  the  Wiener  filter  we  can  simplify 
the  expression  by  noting  from  (1.16)  and  (1.18)  that 


(R5  +  R,,)h  -  =  0 

Therefore,  for  the  Wiener  filter  (III. 27)  simplifies  to 

MSE  =  1  - 

This  result  can  also  be  obtained  directly  using  (1.17). 


(III.  28) 
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Again,  we  have  all  the  necessary  elements  to  plot  the  normalized  mean-square 
error  as  a  function  of  the  model  parameters  and  the  input  signal-to-noise  ratio.  Figure 
31  shows  the  normalized  mean-square  error  plotted  as  a  function  of  a,  for  7  =  0  and  input 
signal-to-noise  ratio  of  0  dB.  It  can  be  seen  that  higher  order  filters  result  in  lower  values 
of  mean-square  error,  as  expected.  However,  for  the  longer  filters,  there  is  significant 
improvement  in  performance  only  for  very  high  values  of  the  signal  correlation  parameter 
a,  i.e.,  the  ability  to  reduce  mean-square  error  depends  heavily  on  the  correlation  present 
in  the  signal. 
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Figure  31.  Mean  Square  Error  for  the  FIR  filter  of  length  P  for  7  =  0  (white  noise)  and 
input  signal-to-noise  ratio  of  0  dB 

3.  Signal  Distortion  for  Higher  Order  FIR  Filter. 

To  derive  an  expression  for  the  signal  distortion  for  the  FIR  filter  of  order  P,  we 
write  the  response  of  the  filter  for  the  signal  alone  using  (1.15)  as  y^ln)  =  s.  Thus, 
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we  have 


E  {s{n)ys{n)}  =  E  {s(n)  s}  =  f,  (IIL29) 


and 

E  {?/5(n)^}  =  E  {h^  s  s"’  h}  =  h’"  R,  h  (111.30) 


Substituting  these  expressions  into  equation  (1.34)  then  produces  the  desired  expression 

(h^  f,)2  1 


SD  =  1  - 


h  a? 


(III.31) 


This  can  be  evaluated  for  the  Wiener  filter  by  substituting  the  solution  (III.25)  for  h. 

Figure  32  shows  the  behavior  of  the  signal  distortion  for  the  FIR  filter  of  different 
orders  P,  for  7  =  0  and  input  signal-to-noise  ratio  of  0  dB.  The  behavior  of  this  measure 
is  similar  to  its  behavior  in  the  case  of  the  HR  filter.  The  signal  distortion  is  0  dB 
for  a  =  drl  and  for  o;  =  7  and  achieves  a  maximum  value  around  a  value  ~  iO.8, 
depending  on  the  length  of  the  filter.  In  addition,  the  value  of  the  signal  distortion  is 
greater  for  larger  filter  lengths. 


C.  SUMMARY 

The  analysis  carried  out  in  this  chapter  for  the  FIR  Wiener  filter  shows  that  this 
filter  has  performance  qualitatively  similar  to  that  of  the  HR  filter,  although  for  the  case 
of  a  first  order  AR  signal  and  noise  model  the  performance,  in  terms  of  the  mean-square 
error  and  processing  gain,  is  never  quite  as  good. 

For  the  first  order  FIR  filter,  which  has  a  single  zero,  the  location  of  the  zero 
changes  from  -Hi  to  —1  as  the  signal  correlation  coefficient  increases  from  —1  to  -Hi. 
When  the  signal  and  noise  have  the  same  correlation  coefficient  (o;  =  7)  the  zero  moves 
to  the  origin  (resultind  in  an  all-pass  filter);  when  both  signal  and  noise  are  perfectly 
correlated  (a  =  7  =  ±1)  the  zero  moves  to  a  location  that  depends  on  the  input  signal- 
to-noise  ratio  (see  (III.  10)).  This  results  in  a  steady  state  estimate  for  the  signal  which 
is  equal  to  a  constant  fraction  of  the  input  observation  sequence. 
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Figure  32.  Signal  Distortion  for  the  FIR  filter  of  length  P  for  7  =  0  (white  noise)  and 
input  signal-to-noise  ratio  of  0  dB 

The  performance  measures  for  the  FIR  filter  approaches  those  of  the  HR  filter  as 
the  order  of  the  filter  increases.  Figures  30,  31,  and  32  show  plots  of  the  three  performance 
measures  for  the  FIR  Wiener  filter  of  orders  2,  4,  and  10.  These  performances  can  be 
compared  to  those  for  the  HR  Wiener  filter  in  Figures  33,  34,  and  35.  A  significant 
diflference  between  the  two  types  of  filters  is  that  the  processing  gain  of  the  FIR  filter, 
when  the  input  signal  is  perfectly  correlated  is  not  infinite,  but  is  limited  to  a  value  P 
(the  filter  length).  Also  the  mean-square  error  does  not  go  to  zero  when  a  =  ±1  as  in 
the  HR  filter  case.  However,  the  signal  distortion  for  the  FIR  filter  does  go  to  zero  for 
a  =  j  and  a  =  ±1,  as  in  the  HR  filter,  and  it  is  not  as  large  as  for  the  HR  filter. 
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Figure  34.  Normalized  mean-square  error  for  the  HR  filter  for  7  =  0  (white  noise)  and 
input  signal-to-noise  ratio  of  0  dB 
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IV.  EXTENDED  OPTIMAL  FILTERING 


The  results  of  the  previous  chapters  show  that  the  Wiener  filter,  although  optimal 
in  the  mean-square  error  sense,  is  not  necessarily  optimal  with  regard  to  minimizing  the 
signal  distortion.  It  will  be  shown  here  that  is  possible  to  create  a  whole  family  of 
filters  related  to  the  Wiener  filter  where  different  performance  measures  are  optimized 
subject  to  various  constraints.  We  begin  with  the  Wiener  filter  itself,  and  then  proceed  to 
discuss  three  possible  variations.  We  then  provide  an  analysis  of  the  performance  of  the 
generalized  FIR  filter  for  the  first  order  AR  signal  and  noise  model,  and  finally  suggest 
how  the  generalized  filter  may  be  used  to  obtain  an  appropriate  trade-off  between  signal 
distortion  and  mean-square  error. 

A.  THE  EXTENDED  OPTIMAL  FILTER  PROBLEM 
1.  Minimizing  mean-square  error 

The  Wiener  filter  simply  optimizes  the  mean-square  error  without  any  additional 
constraints.  We  can  derive  the  FIR  form  of  the  filter  by  beginning  with  (III.27)  which 
applies  to  any  filter.  Then  ignoring  terms  that  do  not  depend  on  h,  we  can  minimize  the 
quantity 

-2h’’f,-hh^R,  h  +  h^R^h 

by  taking  the  gradient  with  respect  to  h  to  obtain  [Ref.  5:  Appendix  A] 

— 2f  j  -j-  2R5h  -J-  2R;jh  =  0 


or 

h  =  (R,  +  R„)-^f,  (IV.l) 

which  is  the  Wiener  filter.  The  minimum  mean-square  error  is  then  given  by  (III. 28). 
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2.  Minimizing  distortion  with  constrained  residual  noise 

Ephraim  and  Van  Trees  [Ref.  7]  have  suggested  a  more  general  filter  for  speech 
enhancement,  based  on  one  measure  of  the  signal  distortion.  Here  we  adapt  this  idea 
using  the  signal  distortion  measure  defined  in  Chapter  I. 

We  begin  with  the  filter  depicted  in  Figure  3,  and  the  responses  ys{n)  and  yr,{n) 
due  to  the  filter  acting  on  the  signal  and  noise  alone.  The  problem  is  to  minimize  the 
distortion  while  constraining  the  residual  noise  power  to  satisfy 

E{yl{n))<al  (IV.2) 

where  aj  is  some  chosen  level  of  residual  noise  power  (not  less  than  the  mean-square  error 
^1^®  Wiener  filter).  We  apply  this  procedure  using  the  signal  distortion  measure 
(1.34).  In  the  context  of  an  FIR  filter,  the  problem  can  be  formulated  as  to  minimize 
(III. 31)  subject  to  the  constraint 


E  {vi(n)}  =  h^R,h  <  <4 


(IV.3) 


Let  us  first  consider  the  procedure  for  minimizing  the  signal  distortion  without 
the  constraint  (IV.3).  To  do  this  it  is  necessary  to  maximize  the  term 


(h^  fs)^  _  fsfjh 
h^R,h  ~  h^R,h 


(IV.4) 


appearing  in  (III.31).  Since  the  solution  is  unique  only  to  within  a  scale  factor,  this  can 
be  done  by  constraining  the  denominator  to  be  any  constant 


h^R,h  =  const  (IV.  5) 

and  maximizing  the  numerator.  Thus,  by  forming  the  Lagrangian  [Ref.  5,  8] 

£  =  h^fsf  Jh  -f  ij,{const  -  h^R^h) 

where  ^  is  a  Lagrange  multiplier,  and  setting  the  gradient  of  the  Lagrangian  to  zero  we 
obtain 

V/i£  =  2f5fJh-2//Rsh  =  0 
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This  shows  that  h  must  satisfy 


fs  f  J  h  =  //  Rs  h  (IV.6) 

i.e.,  h  is  a  generalized  eigenvector  for  the  two  matrices  fj  f  J  and  R5.  The  only  non-trivial 
solution  to  (IV.6)  is  given  by 

h  =  r;'  f, 

which  is  equal  to  the  all-pass  filter  h  =  [1  0  0 . . .0]’"  with  /i  =  fJR^^fs.  Note  that 
the  effect  of  choosing  the  constant  in  (IV.5)  is  merely  to  multiply  the  solution  by  some 
appropriate  scale  factor.  Since  the  scale  factor  is  unimportant  in  this  problem,  we  can 
assume  the  constant  is  chosen  so  that  the  unit  vector  h  =  [1  0  0 ...  0]^  results. 

When  we  introduce  the  constraint  (IV.3)  we  need  to  form  a  new  Lagrangian 

£  =  (h^fs)f  Jh  -i-  (const  -  h^R^h)  -I-  R,,  h)  (IV. 7) 

where  ni  and  ^2  are  Lagrange  multipliers.  Setting  the  gradient  to  zero  then  yields 

V/i£  —  2ts  f  Jh  -  2//iR5h-2)U2R,,h  =  0 

or 

fj  fj  h  = /ii(R5 -H  AR„)h 

where  X  =  1x2/ In  this  case,  we  see  that  the  solution  is  given  by 

h  =  (R,  +  AR„)-^f,  I  (IV.8) 

with  /ii  =  f  J(Rs-|-AR,,)“^fs.  The  parameter  A  can  be  related  to  through  the  constraint 
(IV.2).  In  particular,  substituting  (IV.8)  into  (IV.3)  with  the  equality  yields 

f J  (R,  +  AR„)-i  R„  (R,  +  AR^)-i  f,  = 

which  must  be  solved  by  iteration.  For  the  case  of  white  noise  (R,,  =  a^l)  the  equation 
can  be  simplified  to 

tr  f,  f  J  (R,  -f  =  aHul  (IV.9) 
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The  solution  (IV.8)  for  the  extended  optimal  filter  is  the  same  as  that  of  a  Wiener 
filter  with  modified  noise  power.  Some  particular  special  cases  are  of  interest.  For  A  =  0 
we  obtain  the  all-pass  filter  h  =  [1  0 . . .  0]^  (as  we  have  already  noted),  resulting  in 
the  maximum  residual  noise  power  and  zero  signal  distortion.  For  A  =  1  we 

have  the  Wiener  filter  which  minimizes  the  residual  noise  power  of  =  a^.  By  choosing 
an  appropriate  value  0  <  A  <  1  we  obtain  minimum  distortion  with  residual  noise 
crl  <  al  < 


3.  Minimizing  distortion  for  fixed  processing  gain 

For  this  problem  the  processing  gain  (III.24)  is  constrained  to  a  fixed  value 


h^R,h  J_ 


and  it  is  desired  to  maximize  the  term 


(IV.  10) 


(h^f,)2  ^  h^f.f^h 
h^R,h  h^R,h 

appearing  in  the  signal  distortion  equation  (III.31).  Since  the  solution  for  h  is  only 
unique  to  within  a  scale  factor,  one  can  introduce  the  constraint 


h^Rjh  =  const 


where  as  before,  the  constant  can  be  left  unspecified.  Then  it  is  desired  to  maximize  the 
term  h^f^f  Jh  subject  to  the  additional  constraint 


We  thus  form  the  Lagrangian 


h^R„h  = 


const 
G  Pin 


£  —  h^fsfjh  -I-  (const  —  h^R^h)  -I-  //2(const/Gpi„  —  h^R^h) 
and  set  the  gradient  to  zero  to  obtain 


2?^? Jh  -  2/iiRsh  -  2jU2R„h  =  0 
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or 


fsf  Jh  =  Ati(Rsh  +  AR^)h 

where  A  =  As  before,  the  only  nontrivial  solution  to  this  generalized  eigenvalue 

problem  is  given  by 

h  =  (R5  +  AR„)"^fs 

which  is  of  the  same  form  as  (IV.8).  Note  that  if  the  constant  had  been  specified  in  the 
constraint  introduced  above,  a  scale  factor  would  have  been  required  in  the  last  equation. 
Since  the  scale  factor  is  unimportant,  the  solution  stands  as  specified. 

In  order  to  find  the  appropriate  value  for  A,  we  can  apply  the  original  constraint 
(IV.  10),  writing  it  as 

h^R,h  -  Gpinli^Rnh  =  0 

so  that  A  is  the  solution  to  the  quadratic  equation 


h"'(A)Ah(A)  =  0 


(IV.ll) 


with 

A  =  R,  -  GpinRr,  (IV.  12) 

and  h(A)  is  given  by  (IV.8).  Note  that  G  must  be  chosen  such  that  solutions  exist  and 
the  equation  needs  to  be  solved  by  iteration. 


4.  Maximizing  processing  gain  with  fixed  distortion 

A  final  problem  considered  here  is  to  maximize  the  processing  gain  while  con¬ 
straining  the  distortion  to  be  no  more  than  some  fixed  value  D.  For  this,  the  problem  is 
to  maximize  the  term 

h^R,h 

h^R^h 


subject  to  the  constraint 


(h^  f.)^ 

R,  h 


(IV.13) 
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Again,  the  solution  is  determined  only  to  within  an  arbitrary  scale  factor,  so  we  can 
convert  the  problem  to  one  of  minimizing  the  term  h^R,,h  subject  to  the  two  constraints 

h^R^h  =  const  and  (h^f^)^  =  (1  -  D)a1  ■  const 

Expanding  the  last  term  and  forming  the  Lagrangian  yields 

£  =  h^R,,h  +  (const  —  h^R^h)  +  /^2((1  ~  •  const  —  h^fjf  Jh) 

and  setting  the  gradient  to  zero  produces 

2R^h  —  2)LiiRsh  —  2jU2fsf  Jh  =  0 
or 

fjf  Jh  =  //'(Rjh  +  AR,,)h 

where  [x'  =  —^ij ^2  and  A  =  —l/fxi.  As  before,  this  has  the  solution 

h  =  (Rs  +  AR,,)~^fs 

again  of  the  same  form  as  (IV.8). 

The  implicit  equation  for  A  can  be  obtained  by  writing  the  constraint  (IV.  13)  as 
h’’f,fjh  -  (7^(1  -  £>)h^R,h  =  0 
This  is  of  the  same  form  as  (IV.  11)  with 

A  =  f,f  J  -  a2(l  -  D)R, 


The  foregoing  analyses  show  that  the  solution  to  several  problems  involves  an 
identical  procedure  which  is  equivalent  to  solving  the  Wiener  filtering  problem  with  a 
scale  factor  A  applied  to  the  noise  covariance.  For  each  of  the  problems,  this  scale  factor 
is  determined  by  solving  a  different  nonlinear  equation.  We  note  that  not  all  values  of 
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the  desired  parameter  £>  or  G  in  problems  (3)  and  (4)  will  lead  to  legitimate  solutions; 
we  must  require  at  least  that  0  <  cr^/cr^  <  1,  0  <  G  <  P,  and  0  <  P  <  1.  However 
all  values  for  which  the  parameter  A  is  positive  lead  to  correct  solutions.  Thus,  given 
the  form  of  the  optimal  filter  for  these  various  problems,  it  is  most  efficient  to  start  with 
values  of  A  and  find  the  appropriate  values  of  residual  noise,  distortion,  or  processing 
gain.  An  iterative  search  or  table  lookup  can  then  be  used  to  find  the  appropriate  value 
of  A. 

In  summary,  by  choosing  a  value  for  A  in  (IV.8)  we  can  gain  more  control  over 
the  signal  distortion  at  the  expense  of  increased  residual  noise  and  mean-square  error,  or 
trade  off  processing  gain  for  lower  distortion.  In  the  studies  presented  below,  it  is  shown 
that  this  added  flexibility  has  some  definite  advantages  for  noise  filtering. 

B.  PERFORMANCE  OF  THE  GENERALIZED  FILTER 

By  using  equation  (IV.8)  in  (III.24)  we  can  study  the  processing  gain  of  the  filter 
as  a  function  of  the  parameter  A  and  the  signal  correlation  coefficient  a.  Figure  36  shows 
the  results  of  this  computation  for  7  =  0  (white  noise)  and  an  input  signal-to-noise  ratio 
of  0  dB.  Notice  that  the  processing  gain  is  lower  for  lower  values  of  A,  demonstrating 
that  there  is  less  noise  removal  for  A  <  1. 

We  can  also  substitute  (IV.8)  into  equation  (III.27)  to  obtain  an  expression  for 
the  mean-square  error  as  a  function  of  a  and  A.  Figure  37  shows  these  results  for  the 
same  case  as  Figure  36,  i.e.,  white  noise  and  an  input  signal-to-noise  ratio  of  0  dB.  Again, 
there  is  decrease  in  performance  for  values  of  A  <  1,  going  from  minimum  mean-square 
error  for  A  =  1,  which  corresponds  to  the  optimal  Wiener  filter,  to  maximum  for  A  =  0, 
which  corresponds  to  an  all-pass  filter,  that  does  not  remove  any  noise  from  the  observed 
sequence. 

Finally,  we  can  apply  (IV.8)  to  (III.31)  and  plot  signal  distortion  as  a  function 
of  a  and  A.  Figure  38  shows  the  results  for  the  same  case  as  before.  Here  we  see  that 
the  signal  distortion  of  the  filter  improves  for  higher  values  of  A,  going  from  maximum 
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Figure  36.  Processing  gain  for  the  generalized  Wiener  filter  as  function  of  a  and  A  for 
7  =  0  (white  noise)  and  input  signal-to-noise  ratio  of  0  dB. 

distortion  for  A  =  1  (Wiener  filter)  to  minimum  distortion  for  A  =  0,  where  the  resulting 
all-pass  filter  does  not  alter  the  signal. 

The  rate  of  change  of  the  measures  of  performance  PG,  MSE  and  SD  as  a  func¬ 
tion  of  A  allows  one  to  make  useful  trade-offs.  Notice  from  Figures  36  through  38  that 
for  values  of  A  greater  than  approximately  0.4  the  performance  of  the  filter  in  terms 
of  processing  gain  and  mean-square  error  does  not  change  significantly,  while  the  signal 
distortion  decreases  significantly  over  this  range  of  values  of  A.  This  implies  that  the  gen¬ 
eralized  filter  with  an  appropriately  chosen  value  of  A  may  be  more  desirable  in  practical 
applications  involving  noise  removal  than  the  standard  Wiener  filter.  Since  the  solution 
(IV.8)  for  the  filter  is  equivalent  to  that  of  a  Wiener  filter  with  an  addition  parameter,  it 
is  convenient  to  adapt  this  filter  for  use  in  a  short-time  noise  removal  algorithm.  We  will 
see  in  the  following  chapter  a  practical  application  of  the  new  algorithm,  which  provides 
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Figure  37.  Mean  square  error  for  the  generalized  Wiener  filter  as  function  of  a  and  A  for 
7  =  0  (white  noise)  and  input  signal-to-noise  ratio  of  0  dB 

good  performance  and  more  control  over  the  results. 
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Figure  38.  Signal  distortion  for  the  generalized  Wiener  filter  as  function  of  a  and  A  for 
7  =  0  (white  noise)  and  input  signal-to-noise  ratio  of  0  dB. 
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V.  APPLYING  EXTENDED  OPTIMAL 
FILTERING  TO  UNDERWATER  SIGNALS 


A.  INTRODUCTION 

In  this  chapter  we  apply  the  solution  of  the  extended  optimal  filter  expressed 
in  (IV.8)  to  the  short-time  filtering  algorithm  developed  by  Frack  in  [Ref.  4].  This 
algorithm  assumes  that  both  the  signal  and  the  noise  correlation  functions  {Rs{l)  and 
are  unknown  a  priori;  therefore  they  must  be  estimated  from  the  data  at  hand. 
Since  only  x{n)  =  s{n)+T){n)  is  observed,  however,  only  Rx{l)  can  be  estimated  directly. 
Nevertheless  the  nature  of  the  problem  provides  a  way  to  compute  the  needed  statistics. 
Since  the  signal  is  very  short  (on  the  order  of  seconds  or  milliseconds)  compared  to  the 
time  over  which  the  noise  statistics  are  likely  to  change,  an  estimate  for  Rrf(l)  can  be 
made  from  the  received  data  prior  to  the  onset  of  s{n).  By  using  (1.3),  this  estimate  can 
in  theory  be  subtracted  from  Rx{l)  to  produce  an  estimate  of  Rs{l). 

Although  there  is  no  explicit  analytic  expression  for  the  parameter  A,  we  can 
obtain  the  value  of  A  corresponding  to  any  desired  residual  noise  level  by  solving  (IV.9). 
In  the  case  of  white  noise,  this  is  equivalent  to  writing  (IV. 3)  as 


=  aj  II  h  |p= 

(V.l) 

and  then  (  by  IV.8)  solving  for  A  from 

II  (R.  +  AR,)->f,  |p=  i 

(V.2) 

V 


Therefore,  by  setting  the  level  of  the  residual  noise  that  can  be  tolerated,  we  can 
assign  values  for  A  in  each  segment  by  iterating  on  (V.2)  until  the  equality  is  satisfied, 
thus  obtaining  the  value  of  A  that  minimizes  the  signal  distortion.  Thus,  in  principle,  all 
of  the  quantities  to  perform  the  extended  optimal  filtering  over  a  short  time  interval  can 
be  computed,  and  this  can  be  repeated  over  successive  blocks  of  data. 

The  estimate  of  Rgil)  by  subtraction  of  the  estimated  correlation  functions  is 
not  well  formed  because  there  is  no  guarantee  that  such  an  estimate  will  be  positive 
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y[n]  =  s{n) 


Figure  39.  Prewhitening  in  Short-Time  Extended  Filtering  Algorithm. 

(semi)definite.  This  problem  is  mitigated  if  the  noise  is  white  because  the  procedure 
then  involves  subtraction  of  only  a  single  parameter,  the  white  noise  variance,  from  the 
estimated  Rx  3't  lag  zero.  Further,  the  estimate  of  this  single  parameter  has  lower  variance 
than  the  estimate  of  the  correlation  function  as  a  whole.  Therefore  before  any  further 
steps,  the  entire  data  set  is  processed  by  a  linear  predictive  filter  that  whitens  the  noise. 
After  noise  removal,  the  data  is  processed  by  the  inverse  filter  as  shown  in  Figure  39. 

The  prewhitened  data  is  segmented  into  blocks  where  an  estimate  of  the  local  cor¬ 
relation  function  Rx{l)  is  formed  for  each  segment.  Also,  for  each  segment,  the  algorithm 
finds  the  value  of  A,  since  A  is  a  function  of  the  and  a^.  Extended  optimal 

filtering  is  then  performed  for  each  segment  using  a  filter  designed  for  the  segment,  and 
the  data  is  processed  by  the  inverse  filter  to  undo  the  effects  of  the  prewhitening.  In 
performing  the  filtering,  the  data  is  processed  both  forward  and  backward  through  the 
extended  optimal  filter.  (This  gives  an  approximation  to  a  symmetric  noncausal  filter  of 
twice  the  length.)  Since  the  optimal  filters  are  different  for  each  block,  discontinuities  at 
the  boundaries  can  arise.  The  effect  of  such  discontinuities  can  be  minimized  by  using 
points  from  the  adjacent  segment  to  filter  the  early  points  of  the  current  segment.  In  the 
algorithm,  the  data  is  actually  processed  twice.  The  data  is  first  segmented  and  filtered 
and  the  resulting  frames  are  weighted  by  a  triangular  window  (see  Figure  40).  The  data 
is  then  resegmented  using  frames  shifted  by  half  of  the  frame  length,  filtered  again  and 
weighted  by  a  triangular  window.  The  two  weighted  sets  of  data  are  then  added  to  pro¬ 
duce  the  final  result  and  minimize  any  effects  that  may  occur  at  the  boundaries  between 
frames.  Additional  details  of  this  processing  can  be  found  in  [Ref.  4]. 
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First  P3SS 
filtering  and 
weighting 


3eoond  nass 
filtering  and 
weighting 


Figure  40.  Overlap  Averaging  Technique  Used  in  Noise  Removal. 


B.  RESULTS 

To  obtain  a  quantitative  measure  of  the  signal  distortion,  we  first  need  to  have 
a  clean  signal  s(n),  and  obtain  ys{n),  the  response  of  the  filter  to  this  signal  s(n),  as 
explained  in  Chapter  I.  In  this  way  we  can  demonstrate  our  conclusions  about  the  im¬ 
provement  in  the  signal  distortion  accomplished  by  the  extended  filtering  algorithm. 

For  these  experiments,  an  observation  sequence  x{n)  was  generated  by  adding  a 
white  noise  sequence  with  variance  cr^  to  the  clean  signal  s{n).  The  sequence  x{n)  was 
then  processed  using  the  algorithm  described  above  to  obtain  the  filter  parameters  for 
each  segment  of  this  sequence.  These  same  filter  parameters  were  then  used  to  process 
the  corresponding  segments  of  the  clean  signal  s{n)  to  obtain  ys{n). 

We  can  now  compare  the  effects  of  the  filter  on  the  signal  with  respect  to  signal 
distortion.  In  order  to  do  that,  we  introduce  the  estimates 


5T>  =  1- 


{T.ns{n)  ys{n)Y 
T.nS^{n)  T^nVlin) 


(V.3) 


MSB  =  — ^  (V.4) 

where  the  summation  is  over  all  points  in  all  segments. 

For  all  the  following  experiments,  the  segment  (frame)  length  is  100  points,  the 
filter  order  is  35,  and  the  AR  model  order  for  the  noise  was  arbitrarily  set  to  10  (this 
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parameter  could  be  set  to  any  value  since  the  noise  is  white).  The  first  1,000  points  data, 
consisting  of  noise,  was  used  for  the  noise  sequence. 

Figure  41  shows  the  result  of  filtering  a  clean  synthetically  generated  short  pulse 
signal  with  1,400  points  (41(a)),  contaminated  by  white  noise  with  variance  =  0.36 
(41(b)),  using  the  traditional  Wiener  filter.  The  result,  shown  in  Figure  41(c),  indicates 
that  the  filter  is  effective  in  removing  the  noise.  The  signal  distortion  estimate  (V.3) 
for  this  case  is  SD  =  0.0556,  and  the  normalized  mean-square  error  estimate  (V.4)  is 
MSB  =  0.1896. 

Now,  by  applying  the  extended  filtering  algorithm  for  the  case  where  more  resid¬ 
ual  noise  can  be  tolerated,  we  can  obtain  lower  signal  distortion.  Figure  42  shows  the 
graphical  results  for  this  case  where  the  residual  noise  power  is  set  to  correspond  to  30% 
of  the  input  noise  power  (cr^/a^  =  0.30).  Part  (d)  of  the  figure  shows  the  value  of  A 
that  resulted  for  =  0.30.  Notice  that  A  assumes  values  less  than  1  for  some  of  the 

segments.  The  signal  distortion  estimate  for  this  case  becomes  SD  =  0.0541,  which  is 
an  improvement  of  about  3%,  and  MSE  =  0.1910,  a  degradation  of  about  0.7%,  when 
compared  to  the  traditional  Wiener  filtering  shown  above. 

Figure  43  shows  the  results  for  an  extreme  case  when  the  residual  noise  power  is 
70%  of  the  input  noise  power.  Here  we  can  see  that  A  assumes  very  low  values  for  most 
of  the  segments.  The  signal  distortion  estimation  for  this  case  is  very  low  SD  =  .0358, 
decreasing  the  signal  distortion  by  a  factor  of  36%,  and  MSE  =  0.2249,  a  degradation  of 
about  18%. 

The  following  table  summarizes  the  results  of  these  experiments  using  three  dif¬ 
ferent  settings  for  the  ratio  cr^/a^.  The  values  in  parentheses  indicate  the  increase  or 
decrease  from  the  values  achieved  by  the  Wiener  filter. 

The  results  shown  here  demonstrate  that  when  more  residual  noise  can  be  toler¬ 
ated,  the  signal  distortion  produced  by  the  filter  decreases  significantly  while  the  mean- 
square  error  increases  by  only  a  modest  amount. 

The  noise  removal  algorithm  was  also  applied  to  real  underwater  acoustic  data. 
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estimates 

Wiener 

Filter 

Generalized  Filter 

0.0556 

0.0541 

(-3%) 

MSE 

0.1896 

0.1910 

(+0.7%) 

0.1985 

(+5%) 

0.2249 

(+18%) 

Table  I.  Results  from  the  Generalized  Filter  Experiments. 


Figures  44  and  45  show  the  results  of  applying  the  extended  filtering  technique  to  data 
representing  a  killer  whale  song.  In  Figure  44  we  used  the  traditional  Wiener  filtering 
process,  while  in  Figure  45  we  allowed  the  residual  noise  to  be  20%  of  the  input  noise 
power.  For  this  case,  it  is  diflSicult  to  quantify  the  performance,  since  the  true  underlying 
signal  is  unknown.  However,  upon  listening  to  the  signal,  the  results  are  at  least  com¬ 
parable  to  those  of  the  Wiener  filter  and  the  slight  increase  in  residual  noise  is  the  price 
paid  for  lower  signal  distortion. 
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Figure  41.  Results  of  the  Application  of  the  Extended  Optimal  Filtering  Technique  to 
a  Synthetically  Generated  Short  Pulse  Signal  with  Added  Low  Power  White  Noise,  for 
Minimum  Residual  Noise  (A  =  1).  (a)  Original  Clean  Data,  (b)  Original  Data  plus 
White  Noise,  (c)  Processed  Data. 


(d) 

Figure  42.  Results  of  the  Application  of  the  Extended  Optimal  Filtering  Technique  to 
a  Synthetically  Generated  Short  Pulse  Signal  with  Added  Low  Power  White  Noise,  for 
Residual  Noise  Power  Equivalent  to  30%  of  the  Input  Noise  Power,  (a)  Original  Clean 
Data,  (b)  Original  Data  plus  White  Noise,  (c)  Processed  Data  (d)  Values  of  A  for  Each 
Segment. 
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(c) 


Figure  43.  Results  of  the  Application  of  the  Extended  Optimal  Filtering  Technique  to 
a  Synthetically  Generated  Short  Pulse  Signal  with  Added  Low  Power  White  Noise,  for 
Residual  Noise  Power  Equivalent  to  70%  of  the  Input  Noise  Power,  (a)  Original  clean 
Data,  (b)  Original  Data  plus  White  Noise,  (c)  Processed  Data  (d)  Values  of  A  for  Each 
Segment. 
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Figure  44.  Results  of  the  Application  of  the  Extended  Optimal  Filtering  Technique  to  a 
Killer  Whale  Song  for  Minimum  Residual  Noise  (A  =  1).  (a)  Original  Noisy  Data,  (b) 
Processed  Data. 


VI.  CONCLUSIONS 


A.  SUMMARY 

The  first  three  chapters  of  this  report  provide  an  extensive  analysis  of  the  HR 
and  FIR  Wiener  filters  for  removing  additive  noise  from  a  desired  signal.  Both  signal 
and  noise  are  represented  by  first  order  autoregressive  (AR)  models.  Three  performance 
measures,  namely  processing  gain,  normalized  mean-square  error,  and  signal  distortion 
are  defined  and  evaluated  for  the  Wiener  filters  based  on  these  models.  We  believe  that 
the  behavior  of  these  filters  in  terms  of  the  defined  performance  measures  is  typical  even 
for  other  cases  with  more  general  signal  and  noise  models. 

It  was  seen  that  the  FIR  filter  has  performance  similar  to  that  of  the  HR  filter, 
although  it  does  not  have  the  same  ideal  limiting  characteristics  such  as  infinite  process¬ 
ing  gain  when  the  signal  becomes  perfectly  correlated.  We  also  showed  that  while  both 
forms  of  the  filter  provide  lower  mean-square  error  and  higher  processing  gain  as  the 
signal  becomes  more  correlated,  signal  distortion  is  not  a  monotonic  function  of  correla¬ 
tion.  In  particular,  the  signal  distortion  for  both  forms  of  the  filter  peaks  in  a  region  of 
moderately  high  correlation  and  increases  when  the  input  signal-to-noise  ratio  decreases. 
This  distortion  introduced  by  the  filter  may  be  undesirable  in  some  applications. 

Chapters  IV  and  V  deal  with  a  way  to  improve  the  signal  distortion  resulting 
from  the  Wiener  filter  by  considering  a  more  general  filtering  problem.  The  result  is 
a  generalized  solution  that  can  reduce  the  signal  distortion  of  the  Wiener  filter,  with 
some  increase  in  the  mean-square  error  and  residual  noise.  This  solution  was  adopted 
in  a  short-time  filtering  algorithm  developed  by  Frack  [Ref.  4]  and  tested.  The  results 
illustrate  that  the  extended  filter  provides  more  flexibility  and  allows  more  control  over 
the  signal  distortion  introduced  by  the  filtering  process. 
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B.  FOR  FUTHER  STUDY 

While  extensive  analytical  work  is  presented  in  this  report,  extensive  testing  of 
the  new  algorithms  has  not  been  carried  out.  For  example,  it  would  be  desirable  to 
carry  out  formal  listening  tests  to  see  (over  some  class  of  signals)  if  listeners  preferred 
the  results  of  the  generalized  filter  over  those  of  the  conventional  Wiener  filter  or,  indeed, 
if  they  can  hear  the  difference.  The  greatest  benefits  may  come  in  the  use  of  the  new 
algorithms  in  conjunction  with  further  signal  processing  steps  which  may  be  sensitive  to 
any  distortion  of  the  signal  introduced  by  the  noise  removal  process.  These  are  all  topics 
for  father  study. 
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